Image projection apparatus, image display apparatus, and vehicle

ABSTRACT

A three-dimensional projection apparatus includes a display including a display surface having a plurality of subpixels arranged in a grid along a first direction and a second direction substantially orthogonal to the first direction, an optical element configured to define a light beam direction of an image light emitted from the subpixels for each strip-shaped region of a plurality of strip-shaped regions extending in the second direction on the display surface, an optical member configured to project the image light, the light beam direction of which is defined by the optical element, so that a virtual image of the display surface is formed, and a controller configured to acquire information related to a position of an eye of a subject and to correct, in accordance with the position of the eye, the optical element and an image to be displayed by the display surface.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority to and the benefit of JapanesePatent Application No. 2016-237825, No. 2016-237832, and No. 2016-237877filed Dec. 7, 2016 and Japanese Patent Application No. 2017-132220 filedJul. 5, 2017, the entire contents of which are incorporated herein byreference.

TECHNICAL FIELD

The present disclosure relates to an image projection apparatus, animage display apparatus, a vehicle, a three-dimensional projectionapparatus, and a three-dimensional projection system.

BACKGROUND

Techniques for making a virtual image visible to a subject, such as thedriver of a vehicle, are known.

SUMMARY

An image projection apparatus according to an embodiment of the presentdisclosure includes a display, at least one second optical member, and acontroller. The display is configured to display an image of a polygonas viewed from a virtual camera, the polygon having a texture projectedthereon. The second optical member is configured to project the imageonto the first optical member, and make a virtual image of the imagevisible to a subject. The controller is configured to dynamicallycorrect the polygon in accordance with a position of both eyes of thesubject.

An image display apparatus according to an embodiment of the presentdisclosure includes a display and a controller. The display isconfigured to display an image of a polygon as viewed from a virtualcamera, the polygon having a texture projected thereon. The controlleris configured to dynamically correct the polygon in accordance with aposition of both eyes of a subject.

A vehicle according to an embodiment of the present disclosure includesa first optical member, a display, at least one second optical member,and a controller. The display is configured to display an image of apolygon as viewed from a virtual camera, the polygon having a textureprojected thereon. The second optical member is configured to projectthe image onto the first optical member, and make a virtual image of theimage visible to a subject. The controller is configured to dynamicallycorrect the polygon in accordance with a position of both eyes of thesubject.

A three-dimensional projection apparatus according to an embodiment ofthe present disclosure includes a display, an optical element, anoptical member, and a controller. The display includes a display surfacehaving a plurality of subpixels arranged in a grid along a firstdirection and a second direction substantially orthogonal to the firstdirection. The optical element is configured to define a light beamdirection of an image light emitted from the subpixels for eachstrip-shaped region of a plurality of strip-shaped regions extending inthe second direction on the display surface. The optical member isconfigured to project the image light, the light beam direction of whichis defined by the optical element, so that a virtual image of thedisplay surface is formed. The controller is configured to acquireinformation related to a position of an eye of a subject and to correct,in accordance with the position of the eye, the optical element and animage to be displayed by the display surface.

A three-dimensional projection system according to an embodiment of thepresent disclosure includes a detection apparatus and athree-dimensional projection apparatus. The detection apparatus isconfigured to detect a position of an eye of a subject. Thethree-dimensional projection apparatus includes a display, an opticalelement, an optical member, and a controller. The display includes adisplay surface having a plurality of subpixels arranged in a grid alonga first direction and a second direction substantially orthogonal to thefirst direction. The optical element is configured to define a lightbeam direction of an image light emitted from the subpixels for eachstrip-shaped region of a plurality of strip-shaped regions extending inthe second direction on the display surface. The optical member isconfigured to project the image light, the light beam direction of whichis defined by the optical element, so that a virtual image of thedisplay surface is formed. The controller is configured to acquireinformation related to the position of the eye from the detectionapparatus and to correct, in accordance with the position of the eye,the optical element and an image to be displayed by the display surface.

A vehicle according to an embodiment of the present disclosure includesa three-dimensional projection system. The three-dimensional projectionsystem includes a detection apparatus and a three-dimensional projectionapparatus. The detection apparatus is configured to detect a position ofan eye of a subject. The three-dimensional projection apparatus includesa display, an optical element, an optical member, and a controller. Thedisplay includes a display surface having a plurality of subpixelsarranged in a grid along a first direction and a second directionsubstantially orthogonal to the first direction. The optical element isconfigured to define a light beam direction of an image light emittedfrom the subpixels for each strip-shaped region of a plurality ofstrip-shaped regions extending in the second direction on the displaysurface. The optical member is configured to project the image light,the light beam direction of which is defined by the optical element, sothat a virtual image of the display surface is formed. The controller isconfigured to acquire information related to the position of the eyefrom the detection apparatus, and to correct, in accordance with theposition of the eye, the optical element and an image to be displayed bythe display surface.

An image projection apparatus according to an embodiment of the presentdisclosure includes a display, at least one second optical member, and acontroller. The display is configured to display an image. The secondoptical member is configured to project the image onto the first opticalmember, and make a virtual image of the image visible to a subject. Thecontroller is configured to dynamically correct, in accordance with aposition of both eyes of the subject, the image to be displayed by thedisplay.

An image display apparatus according to an embodiment of the presentdisclosure includes a display and a controller. The display isconfigured to display an image. The controller is configured todynamically correct, in accordance with a position of both eyes of asubject, the image to be displayed by the display.

A vehicle according to an embodiment of the present disclosure includesa first optical member, a display, at least one second optical member,and a controller. The display is configured to display an image. Thesecond optical member is configured to project the image onto the firstoptical member, and make a virtual image of the image visible to asubject. The controller is configured to dynamically correct, inaccordance with a position of both eyes of the subject, the image to bedisplayed by the display.

An image projection apparatus according to an embodiment of the presentdisclosure includes a display, at least one second optical member, and acontroller. The display is configured to display a plurality of images.The second optical member is configured to project the plurality ofimages onto a first optical member, and make virtual images of theplurality of images visible to a subject. The controller is configuredto dynamically correct, in accordance with a position of both eyes ofthe subject, each image of the plurality of images to be displayed bythe display.

An image display apparatus according to an embodiment of the presentdisclosure includes a display and a controller. The display isconfigured to display a plurality of images. The controller isconfigured to dynamically correct, in accordance with a position of botheyes of a subject, each image of the plurality of images to be displayedby the display.

A vehicle according to an embodiment of the present disclosure includesa first optical member, a display, at least one second optical member,and a controller. The display is configured to display a plurality ofimages. The second optical member is configured to project the pluralityof images onto the first optical member, and make virtual images of theplurality of images visible to a subject. The controller is configuredto dynamically correct, in accordance with a position of both eyes ofthe subject, each image of the plurality of images to be displayed bythe display.

An image projection apparatus, an image display apparatus, and a vehicleaccording to an embodiment of the present disclosure increase theconvenience of a technique for making a virtual image visible to asubject.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings:

FIG. 1 illustrates a vehicle and an image projection apparatus accordingto first embodiments of the present disclosure;

FIG. 2 is a block diagram illustrating the schematic configuration ofthe image projection apparatus of FIG. 1;

FIG. 3 is a block diagram illustrating the schematic configuration of animage display apparatus of FIG. 2;

FIG. 4 illustrates an example of correction information corresponding toeach reference position in a vehicle;

FIG. 5 illustrates a first example of reference positions and theposition of both eyes of a subject;

FIG. 6 illustrates a second example of reference positions and theposition of both eyes of a subject;

FIG. 7 is a flowchart illustrating first operations of an imageprojection apparatus;

FIG. 8 is a flowchart illustrating second operations of an imageprojection apparatus;

FIG. 9 is a flowchart illustrating third operations of an imageprojection apparatus;

FIG. 10 illustrates a three-dimensional projection system mounted in avehicle according to second embodiments of the present disclosure;

FIG. 11 illustrates the schematic configuration of the three-dimensionalprojection apparatus illustrated in FIG. 10;

FIG. 12 is a view of a display illustrated in FIG. 11 from the normaldirection of a display surface;

FIG. 13 is a view of a parallax barrier illustrated in FIG. 11 from thenormal direction of a light-blocking surface;

FIG. 14 is a view of the display and the parallax barrier illustrated inFIG. 11 from the parallax barrier side;

FIG. 15 schematically illustrates the relationship between the eyes of asubject and a virtual image;

FIG. 16 illustrates an example of correction information;

FIG. 17 illustrates an example of reference positions and the positionof both eyes of a subject;

FIG. 18 illustrates another example of reference positions and theposition of both eyes of a subject;

FIG. 19 is a flowchart illustrating an example of the processing flowfor the three-dimensional display apparatus illustrated in FIG. 10 todetermine a weighting factor; and

FIG. 20 is a flowchart illustrating an example of the processing flowfor the three-dimensional display apparatus illustrated in FIG. 10 toproject a three-dimensional image.

DETAILED DESCRIPTION

Embodiments of the present disclosure are now described with referenceto the drawings.

First Embodiments

First embodiments of the present disclosure is described. Demand existsfor increasing the convenience of techniques for making a virtual imagevisible to a subject. The present disclosure relates to an imageprojection apparatus, an image display apparatus, and a vehicle thatincrease the convenience of a technique for making a virtual imagevisible to a subject. An image projection apparatus, an image displayapparatus, and a vehicle according to the present disclosure increasethe convenience of a technique for making a virtual image visible to asubject.

Vehicle

A vehicle 10 according to an embodiment of the present disclosure is nowdescribed with reference to FIG. 1. The vehicle 10 includes an imagingapparatus 11 and an image projection apparatus 12.

The “vehicle” in the present disclosure may, for example, encompasswheeled vehicles, ships, and aircraft. Wheeled vehicles may, forexample, include automobiles, industrial wheeled vehicles, railwaywheeled vehicles, wheeled vehicles for daily life, and fixed-wingaircraft that run on a runway. Automobiles may, for example, includepassenger wheeled vehicles, trucks, buses, motorcycles, and trolleybuses. Industrial wheeled vehicles may, for example, include industrialwheeled vehicles for agriculture and for construction. Industrialwheeled vehicles may, for example, include forklifts and golf carts.Industrial wheeled vehicles for agriculture may, for example, includetractors, cultivators, transplanters, binders, combines, and lawnmowers.Industrial wheeled vehicles for construction may, for example, includebulldozers, scrapers, backhoes, cranes, dump cars, and road rollers.Wheeled vehicles may include man-powered wheeled vehicles. The types ofwheeled vehicles are not limited to the above examples. For example,automobiles may include industrial wheeled vehicles that can be drivenon the road. The same wheeled vehicle may also be included in multiplecategories. Ships may, for example, include marine jets, boats, andtankers. Aircraft may, for example, include fixed-wing aircraft androtorcraft.

The imaging apparatus 11 may, for example, include a charge coupleddevice (CCD) image sensor or a complementary metal-oxide semiconductor(CMOS) image sensor. The imaging apparatus 11 can capture a facial imageof a subject 13. The imaging range of the imaging apparatus 11 includesat least an eye box 16, described below. The subject 13 may, forexample, be the driver of the vehicle 10. The imaging apparatus 11 maybe at any position inside or outside of the vehicle 10. For example, theimaging apparatus 11 is located in the dashboard of the vehicle 10. Theimaging apparatus 11 can generate a captured image and output thecaptured image to an external apparatus. The captured image may, forexample, be outputted in a wired or wireless manner, over a controllerarea network (CAN), or the like.

The image projection apparatus 12 can form at least a part of a head-updisplay that makes a virtual image 14 of a required image visible to thesubject 13. The image projection apparatus 12 may be at any positioninside or outside of the vehicle 10. For example, the image projectionapparatus 12 is located in the dashboard of the vehicle 10. The imageprojection apparatus 12 projects an image onto a first optical member 15provided in the vehicle 10. Specifically, the image projection apparatus12 may emit image projection light towards a predetermined region of thefirst optical member 15. Details of the image projection light areprovided below. The first optical member 15 may include a windshield, acombiner, or the like. When the image projection apparatus 12 includesthe first optical member 15, the image projection apparatus 12 can forma head-up display.

Image projection light reflected by the predetermined region of thefirst optical member 15 reaches the eye box 16. The eye box 16 is aregion in actual space in which it is assumed that the eyes of thesubject 13 can be located, taking into consideration the physique,posture, change in posture, and the like of the subject 13, for example.The eye box 16 may have any shape. The eye box 16 may include a flat orthree-dimensional region. The solid arrow in FIG. 1 indicates the pathover which a portion of the image projection light emitted from theimage projection apparatus 12 reaches the eye box 16. The path traveledby light is also referred to as the optical path. At least one of anoptical element that transmits light and an optical element thatreflects light may be included along the optical path. When the eyes ofthe subject 13 are inside the eye box 16 and image projection lightreaches the eye box 16, the subject 13 can see a virtual image 14 of theimage. The virtual image 14 can be visible in front of the vehicle 10,for example.

Image Projection Apparatus

The image projection apparatus 12 is described in detail with referenceto FIG. 2. The image projection apparatus 12 includes an image displayapparatus 17 and one or more second optical members 18. FIG. 2illustrates an example configuration in which the image projectionapparatus 12 includes two second optical members 18 a, 18 b. FIG. 2schematically illustrates an example configuration of the imageprojection apparatus 12. For example, the size, shape, arrangement, andthe like of the image projection apparatus 12 and the constituentelements of the image projection apparatus 12 are not limited to theexample in FIG. 2.

The image display apparatus 17 may be capable of displaying any images.The image display apparatus 17 emits image projection light inside theimage projection apparatus 12. The image projection light may includelight that projects the image displayed by the image display apparatus17. The detailed configuration of the image display apparatus 17 isdescribed below.

The second optical member 18 projects the image displayed by the imagedisplay apparatus 17 onto the first optical member 15 to make a virtualimage 14 of the image visible to the subject 13. Specifically, thesecond optical member 18 causes the image projection light emitted fromthe image display apparatus 17 to reach the outside of the imageprojection apparatus 12. In the example in FIG. 2, the second opticalmembers 18 a, 18 b cause the image projection light emitted from theimage display apparatus 17 to reach the outside of the image projectionapparatus 12. The second optical member 18 may include a lens or amirror. For example, the second optical members 18 a, 18 b may eachinclude a mirror. At least one of the second optical members 18 a, 18 bmay include a lens. One of the second optical members 18 a, 18 b may bea mirror and the other a lens. The solid arrow in FIG. 2 indicates thepath over which a portion of the image projection light emitted from theimage display apparatus 17 is reflected by the second optical members 18a, 18 b, passes through a window provided in the housing of the imageprojection apparatus 12, and reaches the outside of the image projectionapparatus 12. The image projection light that reaches the outside of theimage projection apparatus 12 reaches a predetermined region of thefirst optical member 15 included in the vehicle 10, as illustrated inFIG. 1.

The second optical member 18 may function as a magnifying optical systemthat magnifies the image displayed by the image display apparatus 17.For example, at least one of the second optical members 18 a, 18 b maybe a mirror having a convex shape or a concave shape in at least aportion of the surface that the image projection light reaches. At leastone of the second optical members 18 a, 18 b may be a lens having aconvex shape or a concave shape in at least a portion of the entrancesurface or exit surface of the image projection light. At least aportion of the convex shape and the concave shape may be spherical oraspherical.

Image Display Apparatus

The image display apparatus 17 is described in detail with reference toFIG. 3. The image display apparatus 17 includes a substrate 19, a lightsource element 20, a third optical member 21, a fourth optical member22, a display 23, a communication interface 24, a memory 25, and acontroller 26. The substrate 19, the light source element 20, the thirdoptical member 21, and the fourth optical member 22 may be configured asone light source apparatus 27. In this case, the image display apparatus17 includes the light source apparatus 27, the display 23, thecommunication interface 24, the memory 25, and the controller 26. Thesubstrate 19, the light source element 20, the third optical member 21,the fourth optical member 22, the display 23, the communicationinterface 24, the memory 25, and the controller 26 may be fixed insidethe image display apparatus 17. The light source element 20 may bedisposed on the substrate 19. FIG. 3 schematically illustrates anexample configuration of the image display apparatus 17. For example,the size, shape, arrangement, and the like of the image displayapparatus 17 and the constituent elements of the image display apparatus17 are not limited to the example in FIG. 3.

The light source element 20 may, for example, include one or more lightemission diodes (LEDs), one or more laser apparatuses, or the like. Thelight source element 20 emits light in accordance with control by thecontroller 26. The solid arrow extending from the light source element20 in FIG. 3 indicates the path over which a portion of light emittedfrom the light source element 20 travels. This portion of light emittedfrom the light source element 20 is also simply referred to below aslight from the light source element 20.

With respect to the position of the light source element 20, the thirdoptical member 21 is positioned in the travel direction of light fromthe light source element 20. In the example in FIG. 3, the third opticalmember 21 is positioned to the right of the light source element 20. Thethird optical member 21 includes a collimator lens, for example. Thethird optical member 21 collimates light incident from the light sourceelement 20. The collimated light may be light traveling in a directionsubstantially parallel to the optical axis direction of the thirdoptical member 21.

With respect to the position of the third optical member 21, the fourthoptical member 22 is positioned in the travel direction of light fromthe light source element 20. In the example in FIG. 3, the fourthoptical member 22 is positioned to the right of the third optical member21. The fourth optical member 22 includes a lens, for example. Thefourth optical member 22 may include a Fresnel lens, for example. Thefourth optical member 22 may be fixed inside the image display apparatus17 so that the optical axis of the fourth optical member 22 and theoptical axis of the third optical member 21 are substantially matched.The optical axis of the third optical member 21 or the optical axis ofthe fourth optical member 22 are also referred to as the optical axis ofthe image display apparatus 17 or the optical axis of the light sourceapparatus 27. The direction of travel of image projection light emittedfrom the image display apparatus 17 and the optical axis direction ofthe image display apparatus 17 may be substantially parallel. The fourthoptical member 22 may refract, in a requested direction of travel, atleast a portion of light that passes through and is collimated by thethird optical member 21.

The display 23 may, for example, include a transmission-type liquidcrystal device such as a liquid crystal display (LCD), a micro electromechanical systems (MEMS) shutter display, or the like. With respect tothe position of the fourth optical member 22, the display 23 ispositioned in the travel direction of light from the light sourceelement 20. In the example in FIG. 3, the display 23 is positioned tothe right of the fourth optical member 22. In an embodiment, the lightsource element 20, the third optical member 21, the fourth opticalmember 22, and the display 23 may be disposed along the optical axis ofthe image display apparatus 17 in this order, as illustrated in FIG. 3,for example. The light emitted from the display 23 is also referred toas image projection light below.

The communication interface 24 may include an interface capable ofcommunicating with an external apparatus. The external apparatus may,for example, include an imaging apparatus 11. The “communicationinterface” in the present disclosure may, for example, encompassphysical connectors and wireless communication devices. Physicalconnectors may include an electrical connector corresponding totransmission by an electric signal, an optical connector correspondingto transmission by an optical signal, and an electromagnetic connectorcorresponding to transmission by electromagnetic waves. Electricalconnectors may include connectors conforming to IEC60603, connectorsconforming to the USB standard, connectors comprising RCA terminals,connectors comprising S terminals specified by EIAJ CP-1211A, connectorscomprising D terminals specified by EIAJ RC-5237, connectors conformingto the HDMI® (HDMI is a registered trademark in Japan, other countries,or both) standard, and connectors comprising a coaxial cable thatincludes a BNC connector (British naval connector or baby-series Nconnector). Optical connectors may include a variety of connectorsconforming to IEC 61754. Wireless communication devices may encompasswireless communication devices conforming to standards includingBluetooth® (Bluetooth is a registered trademark in Japan, othercountries, or both) and IEEE 802.11. The wireless communication deviceincludes at least one antenna.

The memory 25 may include a temporary memory device and a secondarymemory device. The memory 25 may, for example, be configured using asemiconductor memory, a magnetic memory, an optical memory, or the like.The semiconductor memory may include volatile memory and non-volatilememory. The magnetic memory may, for example, include a hard disk,magnetic tape, or the like. The optical memory may, for example, includea compact disc (CD), a digital versatile disc (DVD), a Blu-ray® (BD)Disc® (Blu-ray and Blu-ray Disc are registered trademarks in Japan,other countries, or both), or the like. The memory 25 stores variousinformation and programs necessary for operation of the image displayapparatus 17.

For example, the memory 25 may store a plurality of pieces of correctioninformation corresponding to a plurality of positions in actual space.The plurality of positions are also referred to below as referencepositions. Details of the correction information are provided below.FIG. 4 illustrates n pieces of correction information P1-Pncorresponding to n reference positions in actual space. The n referencepositions may be indicated as 3D orthogonal coordinates or 3D polarcoordinates with an arbitrary position in the vehicle 10 as the origin.The reference positions in actual space corresponding to the pieces ofcorrection information may exist inside or outside of the eye box 16.For example, when the eye box 16 is a hexahedron, the plurality ofreference positions may include the positions of the eight vertices ofthe hexahedron. The plurality of reference positions may include thepositions of the eight vertices of a first hexahedron that is largerthan the eye box 16 and includes the eye box 16. The plurality ofreference positions may include the positions of the eight vertices of asecond hexahedron that is smaller than the eye box 16 and is included inthe eye box 16. A plurality of second hexahedrons may be included insidethe eye box 16. The plurality of reference positions may include anyposition within, or on the sides of, the first hexahedron or the secondhexahedron. In an example, the memory 25 may store a plurality of piecesof correction information corresponding to a plurality of referencepositions located inside the eye box 16.

The correction information is now described in detail. As describedabove, the subject 13 can see a virtual image 14 of an image displayedby the display 23 as a result of image projection light being reflectedby the first optical member 15 and reaching the eye box 16. The surfaceshape of the first optical member 15 may, for example, be designed inaccordance with the vehicle 10 to which the first optical member 15 isattached and need not be flat. Furthermore, when the first opticalmember 15 is attached to the vehicle 10, a bending, twisting, or othersuch force is applied to the first optical member 15 due to the rigidityof the vehicle 10, which may cause the first optical member 15 to bedistorted. Therefore, the shape of the virtual image 14 seen by thesubject 13 may be distorted depending on the position of the eyes of thesubject 13 that sees the virtual image 14.

The correction information is used to correct distortion of the shape ofthe virtual image 14 in accordance with the position of the eyes of thesubject 13. The correction information may include information to changethe shape of the image displayed by the display 23 so as to reducedistortion of the shape of the virtual image 14. For example, thecorrection information may include information indicating a shift amountof each of a plurality of feature points on the image displayed by thedisplay 23. The arrangement of the plurality of feature points on theimage may be determined freely. For example, the plurality of featurepoints may be arranged in a grid with freely chosen intervals. Theplurality of feature points may correspond to a plurality of pixelsdisplaying the image. Correction based on the correction informationallows the image displayed by the display 23 to be deformed so that, forexample, the feature points on the image are respectively moved by theshift amounts indicated in the correction information. For example, theimage to be displayed by the display 23 can be deformed evenly orunevenly by correction based on the correction information and thendisplayed by the display 23. The image projection light of the imagedeformed based on the correction information allows the subject 13 tosee a virtual image 14 with reduced distortion. The correctioninformation can be determined by experiment or simulation, for example.

The controller 26 includes one or more processors. The term “processor”encompasses general-purpose processors that execute particular functionsby reading particular programs and dedicated processors that arespecialized for particular processing. The dedicated processor mayinclude an application specific integrated circuit (ASIC). The processormay include a programmable logic device (PLD). The PLD may include afield-programmable gate array (FPGA). The controller 26 may be either asystem-on-a-chip (SoC) or a system in a package (SiP) with one processoror a plurality of processors that work together.

The controller 26 controls overall operations of the light sourceapparatus 27. For example, the controller 26 controls the drive power ofthe light source element 20 to cause the light source element 20 to emitlight. The controller 26 causes the display 23 to display an image. Theimage may include characters or graphics.

The controller 26 acquires a captured image of the subject 13 from theimaging apparatus 11 via the communication interface 24. The controller26 detects the position of both eyes of the subject 13 in actual spacebased on the acquired captured image. Any algorithm using the capturedimage may be adopted to detect the position of both eyes of the subject13 in actual space. For example, the memory 25 stores correspondenceinformation in advance. The correspondence information associates acombination of the position of the face of the subject 13, theorientation of the face, and the size of the face in the captured imagewith the position of both eyes of the subject 13 in actual space. Thecorrespondence information can be determined by experiment orsimulation, for example. The correspondence information may be stored asa lookup table, for example. The controller 26 detects the position ofthe face of the subject 13, the orientation of the face, and the size ofthe face in the captured image. A method using pattern matching or amethod to extract feature points of the subject 13 in the capturedimage, for example, may be adopted to detect the face and eyes. From thecorrespondence information, the controller 26 extracts the position ofboth eyes of the subject 13 in actual space corresponding to thecombination of the position of the face of the subject 13, theorientation of the face, and the size of the face detected in thecaptured image. The controller 26 detects the extracted positions as theposition of both eyes of the subject 13 in actual space.

The controller 26 dynamically corrects the image displayed by thedisplay 23 in accordance with the detected position of both eyes. Thecorrection of the image may, for example, include deformation of theimage. Any algorithm to correct the image may be adopted. Two differentalgorithms are described below in detail.

First Algorithm

Overall, the first algorithm generates third correction informationbased on first correction information corresponding to the position ofthe right eye of the subject 13 and second correction informationcorresponding to the position of the left eye of the subject 13. Thedisplay image displayed by the display 23 is dynamically corrected basedon the generated third correction information. Details are providedbelow.

The controller 26 determines the first correction informationcorresponding to the position of the right eye of the subject 13. Forexample, the controller 26 may determine the first correctioninformation corresponding to the position of the right eye of thesubject 13 by selection from among a plurality of pieces of correctioninformation stored in the memory 25. Alternatively, the controller 26may determine the first correction information corresponding to theposition of the right eye of the subject 13 by generating the firstcorrection information based on two or more pieces of correctioninformation. Any algorithm to generate the first correction informationmay be adopted.

In the example in FIG. 5, the right eye R of the subject 13 ispositioned inside a hexahedron that has eight reference positions asvertices, correction information being associated with each referenceposition. In this case, the controller 26 generates the first correctioninformation corresponding to the position of the right eye R byinterpolation based on two or more pieces of correction informationamong the eight pieces of correction information located near the righteye R.

In the example in FIG. 6, the right eye R of the subject 13 ispositioned outside a hexahedron that has eight reference positions asvertices, correction information being associated with each referenceposition. In this case, the controller 26 generates the first correctioninformation corresponding to the position of the right eye R byextrapolation based on two or more pieces of correction informationamong the eight pieces of correction information located near the righteye R.

The first correction information is generated by interpolation orextrapolation in this configuration. This can reduce the number ofreference positions for which correction information needs to be storedand can reduce the volume of the correction information that needs to bestored in the memory 25.

The controller 26 determines the second correction informationcorresponding to the position of the left eye of the subject 13. Forexample, the controller 26 may determine the second correctioninformation corresponding to the position of the left eye of the subject13 by selection from among a plurality of pieces of correctioninformation stored in the memory 25. Alternatively, the controller 26may determine the second correction information corresponding to theposition of the left eye of the subject 13 by generating the secondcorrection information based on two or more pieces of correctioninformation. The second correction information may be generated in thesame way as the above-described generation of the first correctioninformation. When the left eye L of the subject 13 is positioned insidea hexahedron, as illustrated in the example in FIG. 5, the controller 26generates the second correction information corresponding to theposition of the left eye L by interpolation. When the left eye L of thesubject 13 is positioned outside a hexahedron, as illustrated in theexample in FIG. 6, the controller 26 generates the second correctioninformation corresponding to the position of the left eye L byextrapolation.

The controller 26 generates third correction information based on thefirst correction information and the second correction information. Anyalgorithm to generate the third correction information may be adopted.For example, the controller 26 generates the third correctioninformation by weighting each of the first correction information andthe second correction information. Specifically, a shift amount S3,indicated by the third correction information, of a feature point on theimage is determined by Equation (1) below.S3=α×S1+(1−α)×S2  (1)In Equation (1), α represents a weighting factor having a value of 0 ormore and 1 or less. S1 represents the shift amount of the feature pointas indicated in the first correction information. S2 represents theshift amount of the feature point as indicated in the second correctioninformation. For example, when the weighting factor α is 0.3, the shiftamount S1 is +10 for a certain feature point, and the shift amount S2 is−10 for the feature point, then the shift amount S3 for the featurepoint as indicated in the third correction information isS3=0.3×(+10)+0.7×(−10)=−4.

The weighting factor α may be freely determined in a range of 0 or moreand 1 or less. For example, the weighting factor α may be determined inadvance. Alternatively, the weighting factor α may be determined takinginto consideration the dominant eye of the subject 13.

The determination of the weighting factor α that takes intoconsideration the dominant eye of the subject 13 may, for example, bemade before the subject 13 drives the vehicle 10. Specifically, thecontroller 26 acquires a captured image of the subject 13. Thecontroller 26 detects the position of both eyes of the subject 13 inactual space. The controller 26 determines the first correctioninformation and the second correction information based on the detectedposition of both eyes. The controller 26 sets the weighting factor α toan initial value. The controller 26 generates the third correctioninformation based on the first correction information, the secondcorrection information, and the weighting factor α. The controller 26causes the display 23 to display a reference image corrected using thegenerated third correction information. Any image may be adopted as thereference image. For example, the reference image may include a square,circle, a plurality of lines arranged in a grid, or the like.

The controller 26 changes the weighting factor α in accordance with useroperation by the subject 13, for example. When the weighting factor α ischanged, the controller 26 newly generates the third correctioninformation based on the first correction information, the secondcorrection information, and the changed weighting factor α. Thecontroller 26 causes the display 23 to display a reference imagecorrected using the newly generated third correction information. Theweighting factor α may be changed, and the reference image that iscorrected using the newly generated third correction information may bedisplayed, a plurality of times in accordance with user operation. Thecontroller 26 finalizes the weighting factor α and stores the weightingfactor α in the memory 25 in accordance with user operation by thesubject 13, for example.

When, for example, the subject 13 drives the vehicle 10 after theweighting factor α is finalized as above, the controller 26 uses theposition of both eyes of the subject 13 to dynamically correct thedisplay image that the display 23 is caused to display. Specifically,the controller 26 determines the third correction information based onthe first correction information and the second correction informationcorresponding to the position of both eyes of the subject 13 and thefinalized weighting factor α. The controller 26 dynamically corrects thedisplay image displayed on the display 23 using the determined thirdcorrection information. The display image may include any information orimage. For example, the display image may include a driving supportimage with the travel speed of the vehicle 10, the predicted route, thespeed limit of the road being traveled, sign information, a route guide,or the like; an image indicating obstacles such as pedestrians; oranother such image.

The weighting factor α that allows the subject 13 to perceive the leastamount of distortion in the virtual image 14 differs depending on thevideo composition ratio of the dominant eye and the non-dominant eye ofthe subject 13. By changing the weighting factor α while viewing thevirtual image 14 of the reference image with both eyes, for example, thesubject 13 can cause the weighting factor α that produces the leastdistortion of the virtual image 14 to be stored in the memory 25. Thisconfiguration allows the display image to be corrected using a weightingfactor α that takes into consideration the dominant eye of the subject13. The distortion of the virtual image 14 perceived by the subject 13is therefore reduced, improving visibility of the virtual image 14. Thisimproves the convenience of a technique for making the virtual image 14visible to the subject 13.

Second Algorithm

Overall, the second algorithm determines a specific position based onthe position of both eyes of the subject 13. Third correctioninformation corresponding to the determined specific position is thendetermined. The display image displayed by the display 23 is dynamicallycorrected based on the determined third correction information. Detailsare provided below.

The controller 26 determines the position of both eyes of the subject13. The controller 26 determines a specific position based on theposition of both eyes of the subject 13. Any algorithm to determine thespecific position may be adopted. For example, the controller 26 weightsthe position of the right eye and the position of the left eye of thesubject 13 and determines the specific position. In greater detail, aspecific position Q3 is determined by Equation (2) below.Q3=α×Q1+(1−α)×Q2  (2)In Equation (2), a represents a weighting factor having a value of 0 ormore and 1 or less. Q1 represents the position of the right eye of thesubject 13. Q2 represents the position of the left eye of the subject13. For example, when the weighting factor α is 0.3, the position Q1 ofthe right eye of the subject 13 is {10, 0, 0}, and the position Q2 ofthe left eye of the subject 13 is {20, 10, 0}, then the specificposition Q3 is {17, 7, 0}.

The weighting factor α may be freely determined in a range of 0 or moreand 1 or less. For example, the weighting factor α may be determined inadvance. Alternatively, the weighting factor α may be determined takinginto consideration the dominant eye of the subject 13. The determinationof the weighting factor α that takes into consideration the dominant eyeof the subject 13 may be made in the same way as in the first algorithm.Specifically, the controller 26 acquires a captured image of the subject13. The controller 26 detects the position of both eyes of the subject13 in actual space. The controller 26 sets the weighting factor α to aninitial value. The controller 26 determines a specific position inaccordance with the position of both eyes of the subject 13 and theweighting factor α. The controller 26 determines the third correctioninformation corresponding to the specific position. The determination ofthe third correction information corresponding to the specific positionmay be made in the same way as the determination of the first correctioninformation or the second correction information in the above-describedfirst algorithm. The controller 26 causes the display 23 to display areference image corrected using the determined third correctioninformation.

The controller 26 changes the weighting factor α in accordance with useroperation by the subject 13, for example. When the weighting factor α ischanged, the controller 26 determines a new specific position based onthe position of both eyes of the subject 13 and the changed weightingfactor α. The controller 26 newly determines the third correctioninformation corresponding to the newly determined specific position. Thecontroller 26 causes the display 23 to display a reference imagecorrected using the newly determined third correction information. Theweighting factor α may be changed, and the reference image that iscorrected using the newly determined third correction information may bedisplayed, a plurality of times in accordance with user operation. Thecontroller 26 finalizes the weighting factor α and stores the weightingfactor α in the memory 25 in accordance with user operation by thesubject 13, for example.

When, for example, the subject 13 drives the vehicle 10 after theweighting factor α is finalized as above, the controller 26 uses theposition of both eyes of the subject 13 to dynamically correct thedisplay image that the display 23 is caused to display. In greaterdetail, the controller 26 determines a specific position based on theposition of both eyes of the subject 13 and the finalized weightingfactor α. The controller 26 determines the third correction informationcorresponding to the specific position. The controller 26 dynamicallycorrects the display image displayed on the display 23 using thedetermined third correction information.

Like the first algorithm, the above-described second algorithm allowsthe display image to be corrected using a weighting factor α that takesinto consideration the dominant eye of the subject 13. The distortion ofthe virtual image 14 perceived by the subject 13 is therefore reduced,improving visibility of the virtual image 14. This improves theconvenience of a technique for making the virtual image 14 visible tothe subject 13. With the second algorithm, it suffices to determine thethird correction information corresponding to the specific positiondetermined based on the position of both eyes of the subject 13 and theweighting factor α. Consequently, the processing load can be reduced ascompared to the first algorithm, which generates the third correctioninformation based on the first correction information, the secondcorrection information, and the weighting factor α.

Operations of the image projection apparatus 12 for determining theweighting factor α while taking into consideration the dominant eye ofthe subject 13 are now described with reference to FIG. 7 for aconfiguration adopting the above-described first algorithm. Theseoperations may be performed before the subject 13 drives the vehicle 10,for example.

Step S100: the controller 26 acquires a captured image of the subject 13from the imaging apparatus 11 via the communication interface 24.

Step S101: the controller 26 detects the position of both eyes of thesubject 13 in actual space based on the acquired captured image.

Step S102: the controller 26 determines the first correction informationcorresponding to the position of the right eye and the second correctioninformation corresponding to the position of the left eye of the subject13.

Step S103: the controller 26 sets the weighting factor α to an initialvalue.

Step S104: the controller 26 generates the third correction informationbased on the first correction information, the second correctioninformation, and the weighting factor α.

Step S105: the controller 26 causes the display 23 to display areference image corrected using the third correction information.

Step S106: upon detecting user operation by the subject 13, for example,the controller 26 determines whether to change the weighting factor α.When the controller 26 determines to change the weighting factor α (stepS106: Yes), the process proceeds to step S107. Conversely, when thecontroller 26 determines not to change the weighting factor α (stepS106: No), the process proceeds to step S108.

Step S107: the controller 26 changes the weighting factor α inaccordance with the user operation in step S106, for example.Subsequently, the process returns to step S104.

Step S108: the controller 26 finalizes the weighting factor α inaccordance with the user operation in step S106, for example, and storesthe weighting factor α in the memory 25. The process then terminates.

Operations of the image projection apparatus 12 for dynamicallycorrecting the display image in accordance with the position of botheyes of the subject 13 are now described with reference to FIG. 8 for aconfiguration adopting the above-described first algorithm. Theseoperations may be repeatedly performed while the subject 13 is drivingthe vehicle 10, for example.

Step S200: the controller 26 acquires a captured image of the subject 13from the imaging apparatus 11 via the communication interface 24.

Step S201: the controller 26 detects the position of both eyes of thesubject 13 in actual space based on the acquired captured image.

Step S202: the controller 26 determines the first correction informationcorresponding to the position of the right eye and the second correctioninformation corresponding to the position of the left eye of the subject13.

Step S203: the controller 26 generates the third correction informationbased on the first correction information, the second correctioninformation, and the weighting factor α stored in the memory 25.

Step S204: the controller 26 causes the display 23 to display thedisplay image corrected using the third correction information.Subsequently, the process returns to step S200.

Operations of the image projection apparatus 12 for dynamicallycorrecting the display image in accordance with the position of botheyes of the subject 13 are now described with reference to FIG. 9 for aconfiguration adopting the above-described second algorithm. Theseoperations may be repeatedly performed while the subject 13 is drivingthe vehicle 10, for example.

Step S300: the controller 26 acquires a captured image of the subject 13from the imaging apparatus 11 via the communication interface 24.

Step S301: the controller 26 detects the position of both eyes of thesubject 13 in actual space based on the acquired captured image.

Step S302: the controller 26 determines a specific position based on theposition of the right eye and the position of the left eye of thesubject 13 and the weighting factor α that is stored in the memory 25.

Step S303: the controller 26 determines the third correction informationcorresponding to the determined specific position.

Step S304: the controller 26 causes the display 23 to display thedisplay image corrected using the third correction information.Subsequently, the process returns to step S300.

In accordance with the position of both eyes of the subject 13, theimage projection apparatus 12 according to an embodiment dynamicallycorrects the image that the display 23 is caused to display, asdescribed above. This configuration reduces the distortion of thevirtual image 14 perceived by the subject 13, thereby improvingvisibility of the virtual image 14. This improves the convenience of atechnique for making the virtual image 14 visible to the subject 13.

The image projection apparatus 12 may store a plurality of pieces ofcorrection information corresponding to a plurality of referencepositions in actual space. The image projection apparatus 12 maydetermine first correction information corresponding to the position ofthe right eye of the subject 13 and second correction informationcorresponding to the position of the left eye of the subject 13 based ona stored plurality of pieces of correction information. The imageprojection apparatus 12 may store a weighting factor that is determinedtaking into consideration the dominant eye of the subject 13. The imageprojection apparatus 12 may generate the third correction informationbased on the first correction information, the second correctioninformation, and the weighting factor. The image projection apparatus 12may use the third correction information to correct the image that thedisplay 23 is caused to display. This configuration accurately reducesthe distortion of the virtual image 14 perceived by the subject 13,thereby further improving visibility of the virtual image 14.

The image projection apparatus 12 may determine the specific positionbased on the position of the right eye of the subject 13, the positionof the left eye of the subject 13, and the weighting factor that isdetermined taking into consideration the dominant eye of the subject 13.The image projection apparatus 12 may determine the third correctioninformation corresponding to the specific position based on a pluralityof pieces of correction information stored in the memory 25. The imageprojection apparatus 12 may use the third correction information tocorrect the image that the display 23 is caused to display. Thisconfiguration accurately reduces the distortion of the virtual image 14perceived by the subject 13, thereby further improving visibility of thevirtual image 14.

Second Embodiments

Second embodiments of the present disclosure is described. Demand existsfor increasing the convenience of techniques for making a virtual imageof a three-dimensional image visible to a subject. It is an objective ofthe present embodiments to provide a three-dimensional projectionapparatus, a three-dimensional projection system, and a vehicle thatincrease the convenience of a technique for making a virtual image of athree-dimensional image visible to a subject. The present embodimentscan increase the convenience of techniques for making a virtual image ofa three-dimensional image visible to a subject.

Vehicle

A vehicle 100 according to the present embodiments is now described withreference to FIG. 3. The vehicle 100 can be mounted in athree-dimensional projection system 1000. The three-dimensionalprojection system 1000 includes a detection apparatus 11 and athree-dimensional projection apparatus 12. In FIG. 3, the directionconnecting the eyes of the subject is the x-axis direction, the verticaldirection is the y-axis direction, and the direction orthogonal to thex-axis direction and the y-axis direction is the z-axis direction.

A detection apparatus 110 can detect the position of the eyes of asubject 130. The detection apparatus 110 can transmit the detectedposition of the eyes to the three-dimensional projection apparatus 120.The detection apparatus 110 may be at any position inside or outside ofthe vehicle 100. For example, the detection apparatus 110 may be locatedin the dashboard of the vehicle 100. The detection apparatus 110 mayoutput information indicating the position of the eyes to thethree-dimensional projection apparatus 12 in a wired or wireless manner,over a controller area network (CAN), or the like.

The detection apparatus 110 may include an imaging apparatus. Theimaging apparatus may, for example, include a CCD image sensor or a CMOSimage sensor. The imaging apparatus can capture a facial image of thesubject 130. The imaging range of the imaging apparatus includes atleast an eye box 160, described below. The subject 130 may, for example,be the driver of the vehicle 100.

The detection apparatus 110 can detect the position of both eyes of thesubject 130 in actual space based on the captured image generated by theimaging apparatus. Any algorithm using the captured image may be adoptedto detect the position of both eyes of the subject 130 in actual space.For example, the detection apparatus 110 can store correspondenceinformation in advance. The correspondence information associates acombination of the position of the face of the subject 130, theorientation of the face, and the size of the face in the captured imagewith the position of both eyes of the subject 130 in actual space. Thecorrespondence information can be determined by experiment orsimulation, for example. The correspondence information may be stored asa lookup table, for example. The detection apparatus 110 can detect theposition of the face of the subject 130, the orientation of the face,and the size of the face in the captured image. A method using patternmatching or a method to extract feature points of the subject 130 in thecaptured image, for example, may be adopted to detect the face and eyes.From the correspondence information, the detection apparatus 110 canextract the position of both eyes of the subject 130, in actual space,that can correspond to the combination of the position of the face ofthe subject 130, the orientation of the face, and the size of the facedetected in the captured image. The detection apparatus 110 can detectthe extracted positions as the position of both eyes of the subject 130in actual space.

Instead of including an imaging apparatus, the detection apparatus 110may be connected to an imaging apparatus. The detection apparatus 110may include an input terminal for input of a signal from the imagingapparatus. In this case, the imaging apparatus may be connected directlyto the input terminal. The detection apparatus 110 may be connectedindirectly to the input terminal over a shared network. The detectionapparatus 110 may detect the position of the eyes of the subject 130from a video signal inputted to the input terminal.

The detection apparatus 110 may, for example, include a sensor. Thesensor may be an ultrasonic sensor, an optical sensor, or the like. Thedetection apparatus 110 may detect the position of the head of thesubject 130 with the sensor and detect the position of the eyes of thesubject 130 based on the position of the head. The detection apparatus110 may use two or more sensors to detect the position of the eyes ofthe subject 130 as coordinates in three-dimensional space.

The three-dimensional projection apparatus 120 makes a virtual image 140visible to the subject 130 of the vehicle 100, for example. The virtualimage 140 may include a first virtual image 140 a and a second virtualimage 140 b. The first virtual image 140 a is a virtual image displayedby a display 200, described below. The second virtual image 140 b is avirtual image of a parallax barrier 210, described below. Thethree-dimensional projection apparatus 120 in the present embodimentscan emit image light, described below, towards a predetermined region ofa first optical member 150 included in the vehicle 100. The emittedimage light is reflected in the predetermined region of the firstoptical member 150 and reaches the eyes of the subject 130. Thethree-dimensional projection apparatus 120 can function as a head-updisplay in this way. In the present embodiments, the first opticalmember 150 may be a windshield. In another embodiment, the first opticalmember 150 may be a combiner.

The three-dimensional projection apparatus 120 may be at any positioninside or outside of the vehicle 100. For example, the three-dimensionalprojection apparatus 120 may be located in the dashboard of the vehicle100. The three-dimensional projection apparatus 120 can project an imageonto the first optical member 150 provided in the vehicle 100.Specifically, the three-dimensional projection apparatus 120 may emitimage light towards a predetermined region of the first optical member150.

The first optical member 150 can reflect the image light from thethree-dimensional projection apparatus 120 in the predetermined region.Image light reflected in the predetermined region of the first opticalmember 150 reaches the eye box 160. The eye box 160 is a region inactual space in which it is assumed that the eyes of the subject 130 canbe located, taking into consideration the physique, posture, change inposture, and the like of the subject 130, for example. The eye box 160may have any shape. The eye box 160 may include a flat orthree-dimensional region. The solid arrow in FIG. 3 indicates the pathover which a portion of the image projection light emitted from thethree-dimensional projection apparatus 120 reaches the eye box 160. Thepath traveled by image light is also referred to as the optical path.When the eyes of the subject 130 are inside the eye box 160 and imagelight reaches the eye box 160, the subject 130 can see the virtual image140. The virtual image 140 can be visible in front of the vehicle 100,for example.

The first optical member 150 may include a windshield, a combiner, orthe like. When the three-dimensional projection apparatus 120 includesthe first optical member 150, the three-dimensional projection apparatus120 can form a head-up display.

Image Projection Apparatus

The three-dimensional projection apparatus 120 is described in detailwith reference to FIG. 4. The three-dimensional projection apparatus 120includes a three-dimensional display apparatus 170 and one or moresecond optical members 180 (optical member). FIG. 4 illustrates anexample configuration in which the three-dimensional projectionapparatus 120 includes two second optical members 180 a, 180 b. FIG. 4schematically illustrates an example configuration of thethree-dimensional projection apparatus 120. For example, the size,shape, arrangement, and the like of the three-dimensional projectionapparatus 120 and the constituent elements of the three-dimensionalprojection apparatus 120 are not limited to the example in FIG. 4.

Second Optical Member

The second optical member 180 projects an image, displayed on a displaysurface 2010 of the display 200 included in the three-dimensionaldisplay apparatus 170, onto the first optical member 150 to make a firstvirtual image 140 a of the image visible to the subject 130. The secondoptical member 180 projects the parallax barrier 210 included in thethree-dimensional display apparatus 170 onto the first optical member150 to make a second virtual image 140 b of the parallax barrier 210visible to the subject 130.

Specifically, the second optical member 180 causes the light emittedfrom the three-dimensional display apparatus 170 to reach the outside ofthe three-dimensional projection apparatus 120. In the example in FIG.4, the second optical members 180 a, 180 b cause the light emitted fromthe three-dimensional display apparatus 170 to reach the outside of thethree-dimensional projection apparatus 120. The second optical member180 may include a lens or a mirror. For example, the second opticalmembers 180 a, 180 b may each include a mirror. At least one of thesecond optical members 180 a, 180 b may include a lens. One of thesecond optical members 180 a, 180 b may be a mirror and the other alens. The dashed dotted arrow in FIG. 4 indicates the path over which aportion of the light emitted from the three-dimensional displayapparatus 170 is reflected by the second optical members 180 a, 180 b,passes through a window provided in the housing of the three-dimensionalprojection apparatus 120, and reaches the outside of thethree-dimensional projection apparatus 120. The light reaching theoutside of the three-dimensional projection apparatus 120 can reach thepredetermined region of the first optical member 150.

The second optical member 180 may function as a magnifying opticalsystem that magnifies the image displayed by the three-dimensionaldisplay apparatus 170. For example, at least one of the second opticalmembers 180 a, 180 b may be a mirror having a convex shape or a concaveshape in at least a portion of the surface that the light reaches. Atleast one of the second optical members 180 a, 180 b may be a lenshaving a convex shape or a concave shape in at least a portion of theentrance surface or exit surface of the light. At least a portion of theconvex shape and the concave shape may be spherical or aspherical.

Three-Dimensional Display Apparatus

The three-dimensional display apparatus 170 is described in detail withreference to FIG. 4. The three-dimensional display apparatus 170 may belocated inside the three-dimensional projection apparatus 120. Thethree-dimensional display apparatus 170 includes an irradiator 190, thedisplay 200, the parallax barrier 210 (optical element), a communicationinterface 220, a memory 230, and a controller 240. FIG. 4 schematicallyillustrates an example configuration of the three-dimensional displayapparatus 170. For example, the size, shape, arrangement, and the likeof the three-dimensional display apparatus 170 and the constituentelements of the three-dimensional display apparatus 170 are not limitedto the example in FIG. 4.

The display 200 is a display device. Any display panel, such as atransmission-type liquid crystal display panel, can be adopted as thedisplay 200. As illustrated in FIG. 5, the display 200 has a pluralityof regions partitioned on the plate-like display surface 2010 in a firstdirection and a second direction substantially orthogonal to the firstdirection. The first direction may be referred to as the horizontaldirection. The second direction may be referred to as the verticaldirection. The first and second directions are not, however, limited tothese examples. In the drawings, the first direction is represented asthe u-axis direction and the second direction as the v-axis direction.The u-axis direction of the image displayed by the display surface 2010can correspond to the x-axis direction in the first virtual image 140 aof the image.

One subpixel corresponds to each partitioned region. Accordingly, thedisplay surface 2010 includes a plurality of subpixels arranged in agrid along the horizontal direction and the vertical direction. Eachsubpixel corresponds to one of the colors R, G, B. The combination ofthe three subpixels R, G, B can form one pixel. The horizontal directionis, for example, the direction in which the plurality of subpixelsforming one pixel are aligned. The vertical direction is, for example,the direction in which subpixels of the same color are aligned. Thedisplay 200 is not limited to a transmission-type liquid crystal panel.Another display panel, such as an organic EL display panel, can beadopted. When a self-luminous display panel is adopted as the display200, the three-dimensional display apparatus 170 need not include theirradiator 190.

As described above, a plurality of subpixels arranged on the displaysurface 2010 form a subpixel group Pg. The subpixel group Pg includes aright subpixel group Pgr (first subpixel group) and a left subpixelgroup Pgl (second subpixel group). The right subpixel group Pgr and theleft subpixel group Pgl are arranged next to each other in thehorizontal direction. A plurality of subpixel groups Pg are repeatedlyarranged next to each other in the horizontal direction.

The right subpixel group Pgr includes subpixels in predetermined lines.Specifically, the right subpixel group Pgr includes a predeterminedconsecutive number, in the horizontal direction, of vertical lines ofcontinuous subpixels displaying a right eye image (first image). Theleft subpixel group Pgl includes subpixels in predetermined lines.Specifically, the left subpixel group Pgl includes a predeterminedconsecutive number, in the horizontal direction, of vertical lines ofcontinuous subpixels displaying a left eye image (second image). Theright eye image is an image to be made visible to the right eye (firsteye) of the subject 130. The left eye image is an image to be madevisible to the left eye (second eye) of the subject 130.

In the example illustrated in FIG. 5, a left subpixel group Pgl thatincludes four consecutive lines, in the horizontal direction, ofsubpixels is arranged on the display surface 2010. A right subpixelgroup Pgr that includes four consecutive lines, in the horizontaldirection, of subpixels is arranged on the display surface 2010 adjacentto the left subpixel group Pgl in the horizontal direction.

As illustrated in FIG. 6, the parallax barrier 210 can define the lightbeam direction, which is the propagation direction of an image lightemitted from the subpixels, for each of a plurality of open regions 210b, which are strip-shaped regions extending in the vertical direction.The regions, on the display surface 2010, containing the subpixels thatemit image light reaching the eyes of the subject 130 are determined bythe parallax barrier 210 prescribing the light beam direction of animage light emitted from the subpixels.

Specifically, the parallax barrier 210 is formed by a flat surface alongthe display surface 2010, as illustrated in FIG. 4. The parallax barrier210 is arranged at a predetermined distance (gap) g from the displaysurface 2010. The parallax barrier 210 can be positioned on the oppositeside of the display 200 from the irradiator 190. The parallax barrier210 can be positioned on the irradiator 190 side of the display 200.

As illustrated in FIG. 6, the parallax barrier 210 includes a pluralityof light-blocking surfaces 210 a that block the image light. Theplurality of light-blocking surfaces 210 a can define the open region210 b between adjacent light-blocking surfaces 210 a. The open regions210 b have a higher light transmittance than the light-blocking surfaces210 a. The light-blocking surfaces 210 a have a lower lighttransmittance than the open regions 210 b. The open regions 210 b areportions for transmitting light that can be incident on the parallaxbarrier 210. The open regions 210 b may transmit light at atransmittance of a first predetermined value or greater. The firstpredetermined value may, for example, be 100% or a value near 100%. Thelight-blocking surfaces 210 a are non-light transmissive portions forblocking light that can be incident on the parallax barrier 210. Inother words, the light-blocking surfaces 210 a block the image displayedon the display 200. The light-blocking surfaces 210 a may block light ata transmittance of a second predetermined value or less. The secondpredetermined value may, for example, be 0% or a value near 0%.

The open regions 210 b and the light-blocking surfaces 210 a extend inthe vertical direction along the display surface 2010 and are arrangedto alternate in the horizontal direction. The open regions 210 b candefine the light beam direction of an image light emitted from thesubpixels.

The parallax barrier 210 may be configured by a liquid crystal shutter.The liquid crystal shutter can control the transmittance of light inaccordance with an applied voltage. The liquid crystal shutter may beconfigured by a plurality of pixels and control the transmittance oflight in each pixel. The liquid crystal shutter can be formed so thatregions with high transmittance of light or regions with lowtransmittance of light have any shape. When the parallax barrier 210 isformed by a liquid crystal shutter, the open regions 210 b may beregions having a transmittance of the first predetermined value orgreater. When the parallax barrier 210 is formed by a liquid crystalshutter, the light-blocking surfaces 210 a may be regions having atransmittance of the second predetermined value or less.

A portion of the image light emitted from the display surface 2010 ofthe display 200 can be transmitted by the parallax barrier 210 and reachthe first optical member 150 through the second optical members 180 a,180 b. The image light may then be reflected by the first optical member150 and reach the eyes of the subject 130. Consequently, the eyes of thesubject 130 can recognize the first virtual image 140 a of the display200 in front of the first optical member 150. In the presentapplication, the front is the direction of the first optical member 150as seen from the subject 130. The front is the direction in which thevehicle 100 normally moves. The parallax barrier 210 forms the secondvirtual image 140 b in front of the first optical member 150 on thefirst optical member 150 side of the first virtual image 140 a. Asillustrated in FIG. 7, the subject 130 can see an image in which thedisplay 200 appears to be at the position of the first virtual image 140a and the parallax barrier 210 appears to be at the position of thesecond virtual image 140 b.

FIG. 7 illustrates the subpixels of the first virtual image 140 a, ofthe display 200, that can be observed by the subject 130. Subpixelslabeled L display a virtual image of the left eye image. Subpixelslabeled R display a virtual image of the right eye image. Furthermore,FIG. 7 illustrates a left eye visible region 140 a a and a left eyeinvisible region 140 a b of the first virtual image 140 a observed fromthe left eye of the subject 130.

A more detailed description is provided now with reference to FIG. 8.The visible regions 140 a a are regions of the first virtual image 140 avisible to the left eye as a result of an image light being emitted fromthe display surface 2010, passing through the open region 210 b of theparallax barrier 210, and reaching the left eye. The invisible regions140 a b are regions of the first virtual image 140 a in which imagelight is not visible to the left eye, due to the image light emittedfrom the display surface 2010 being blocked by the light-blockingsurfaces 210 a of the parallax barrier 210. At this time, the right eyeof the user does not see the left eye visible regions 140 a a in thefirst virtual image 140 a. The right eye of the user can see the lefteye invisible regions 140 a b that display a virtual image of the righteye image in the first virtual image 140 a. Accordingly, the right eyeof the subject 130 sees the right eye image but does not see the lefteye image.

In this way, the right eye of the subject 130 can see the right eyeimage. The left eye of the subject 130 can see the left eye image.Accordingly, when the left eye image and the right eye image haveparallax, the subject 130 can see a three-dimensional image.

The communication interface 220 may include an interface capable ofcommunicating with an external apparatus. The external apparatus may,for example, include the detection apparatus 110.

The memory 230 may include a temporary memory device and a secondarymemory device. The memory 230 may, for example, be configured using asemiconductor memory, a magnetic memory, an optical memory, or the like.The memory 230 can store various information and programs necessary foroperation of the three-dimensional display apparatus 170.

For example, the memory 230 may store a plurality of pieces ofcorrection information corresponding to a plurality of positions inactual space. The plurality of positions are also referred to below asreference positions. Details of the correction information are providedbelow. FIG. 9 illustrates n pieces of correction information P1 to Pncorresponding to n reference positions in actual space. The n referencepositions may be indicated as 3D orthogonal coordinates or 3D polarcoordinates with an arbitrary position in the vehicle 100 as the origin.The reference positions in actual space corresponding to the pieces ofcorrection information may exist inside or outside of the eye box 160.For example, when the eye box 160 is a hexahedron, the plurality ofreference positions may include the positions of the eight vertices ofthe hexahedron. The plurality of reference positions may include thepositions of the eight vertices of a first hexahedron that is largerthan the eye box 160 and that can include the eye box 160. The pluralityof reference positions may include the positions of the eight verticesof a second hexahedron that is smaller than the eye box 160 and isincluded in the eye box 160. A plurality of second hexahedrons may beincluded inside the eye box 160. The plurality of reference positionsmay include any position within, or on the sides of, the firsthexahedron or the second hexahedron. In an example, the memory 230 maystore a plurality of pieces of correction information corresponding to aplurality of reference positions located inside the eye box 160.

The correction information is now described in detail. As describedabove, the subject 130 can see a virtual image 140 of an image displayedby the display 200 as a result of an image light being reflected by thefirst optical member 150 and reaching the eye box 160. The surface shapeof the first optical member 150 may, for example, be designed inaccordance with the vehicle 100 to which the first optical member 150 isattached and need not be flat. Furthermore, when the first opticalmember 150 is attached to the vehicle 100, a bending, twisting, or othersuch force is applied to the first optical member 150 due to therigidity of the vehicle 100, which may cause the first optical member150 to be distorted. Therefore, the shape of the virtual image 140 seenby the subject 130 may be distorted depending on the position of theeyes of the subject 130 that sees the virtual image 140.

The correction information is used to correct distortion of the shape ofthe first virtual image 140 a and the second virtual image 140 b inaccordance with the position of the eyes of the subject 130. Thecorrection information may include information to change the shape ofthe image displayed by the display 200 so as to reduce distortion of theshape of the first virtual image 140 a and the second virtual image 140b. For example, the correction information may include informationindicating a shift amount of each of a plurality of feature points onthe image displayed by the display 200. The arrangement of the pluralityof feature points on the image may be determined freely. For example,the plurality of feature points may be arranged in a grid with freelychosen intervals. The plurality of feature points may correspond to aplurality of pixels displaying the image. Correction based on thecorrection information allows the image displayed by the display 200 tobe deformed so that, for example, the feature points on the image arerespectively moved by the shift amounts indicated in the correctioninformation. For example, the image to be displayed by the display 200can be deformed evenly or unevenly by correction based on the correctioninformation and then displayed by the display 200. The image light ofthe image deformed based on the correction information allows thesubject 130 to see a virtual image 140 with reduced distortion. Thecorrection information can be determined by experiment or simulation,for example.

The controller 240 includes one or more processors. The term “processor”encompasses general-purpose processors that execute particular functionsby reading particular programs and dedicated processors that arespecialized for particular processing. The controller 240 may be eitheran SoC or an SiP with one processor or a plurality of processors thatwork together.

The controller 240 can control operations of the irradiator 190, thedisplay 200, and the parallax barrier 210. For example, the controller240 can control the drive power of the irradiator 190 to cause theirradiator 190 to emit light. The controller 240 can cause the display200 to display an image. The image may include characters or graphics.The controller 240 can control the voltage applied to the liquid crystalshutter forming the parallax barrier 210 to control the transmittance oflight in the parallax barrier 210. The processing of the controller 240to control the display 200 and the parallax barrier 210 is describedbelow in detail.

The controller 240 can acquire information related to the position ofthe eyes of the subject 130 from the detection apparatus 110 through thecommunication interface 220. The controller 240 can acquire a capturedimage from an imaging apparatus included in the detection apparatus 110.In this case, the controller 240 can detect the position of both eyes ofthe subject 130 in actual space based on the acquired captured image.

The controller 240 can dynamically correct the right eye image and theleft eye image, displayed by the display 200, in accordance with the eyepositions indicated by the acquired information. The controller 240 candynamically correct the parallax barrier 210 in accordance with the eyepositions. Any algorithm to correct the image may be adopted. A specificexample of the controller 240 using an algorithm to correct an image isdescribed below.

Overall, the controller 240 can use an algorithm to determine firstcorrection information corresponding to the position of the right eye ofthe subject 130 and second correction information corresponding to theposition of the left eye of the subject 130. The controller 240 cancorrect the right eye image based on the first correction information.The controller 240 can correct the left eye image based on the secondcorrection information. The controller 240 can generate third correctioninformation based on the first correction information and the secondcorrection information. The controller 240 can dynamically correct theparallax barrier 210 based on the generated third correctioninformation. Details are provided below.

The controller 240 can determine the first correction informationcorresponding to the position of the right eye of the subject 130. Forexample, the controller 240 may determine the first correctioninformation by selection from among a plurality of pieces of correctioninformation stored in the memory 230. Alternatively, the controller 240may determine the first correction information by generating the firstcorrection information based on two or more pieces of correctioninformation. Any algorithm to generate the first correction informationmay be adopted.

In the example in FIG. 10, the right eye E_(R) of the subject 130 ispositioned inside a hexahedron that has eight reference positions asvertices, correction information being associated with each referenceposition. In this case, the controller 240 can generate the firstcorrection information corresponding to the position of the right eyeE_(R) by interpolation based on two or more pieces of correctioninformation among the eight pieces of correction information locatednear the right eye E_(R).

In the example in FIG. 11, the right eye E_(R) of the subject 130 ispositioned outside a hexahedron that has eight reference positions asvertices, correction information being associated with each referenceposition. In this case, the controller 240 can generate the firstcorrection information corresponding to the position of the right eyeE_(R) by extrapolation based on two or more pieces of correctioninformation among the eight pieces of correction information locatednear the right eye E_(R).

The first correction information is generated by interpolation orextrapolation in this configuration. This can reduce the number ofreference positions for which correction information needs to be storedand can reduce the volume of the correction information that needs to bestored in the memory 230.

The controller 240 can determine the second correction informationcorresponding to the position of the left eye of the subject 130. Forexample, the controller 240 may determine the second correctioninformation corresponding to the position of the left eye of the subject130 by selection from among a plurality of pieces of correctioninformation stored in the memory 230. Alternatively, the controller 240may determine the second correction information corresponding to theposition of the left eye of the subject 130 by generating the secondcorrection information based on two or more pieces of correctioninformation. The second correction information may be generated in thesame way as the above-described generation of the first correctioninformation. When the left eye E_(L) of the subject 130 is positionedinside a hexahedron, as illustrated in the example in FIG. 10, thecontroller 240 can generate the second correction informationcorresponding to the position of the left eye E_(L) by interpolation.When the left eye E_(L) of the subject 130 is positioned outside ahexahedron, as illustrated in the example in FIG. 11, the controller 240can generate the second correction information corresponding to theposition of the left eye E_(L) by extrapolation.

The controller 240 can generate third correction information based onthe first correction information and the second correction information.Any algorithm to generate the third correction information may beadopted. For example, the controller 240 can generate the thirdcorrection information by weighting each of the first correctioninformation and the second correction information. Specifically, a shiftamount S3, indicated by the third correction information, of a featurepoint on the image is determined by the above-described Equation (1).

The weighting factor α may be freely determined in a range of 0 or moreand 1 or less. For example, the weighting factor α may be determined inadvance. Alternatively, the weighting factor α may be determined takinginto consideration the dominant eye of the subject 130.

The determination of the weighting factor α that takes intoconsideration the dominant eye of the subject 130 may, for example, bemade before the subject 130 drives the vehicle 10. A method ofdetermining the weighting factor α is described below. The controller240 can acquire information related to the position of both eyes fromthe detection apparatus 110. The controller 240 can determine the firstcorrection information and the second correction information on thebasis the position of both eyes indicated by the acquired information.The controller 240 can set the weighting factor α to an initial value.The controller 240 can generate the third correction information basedon the first correction information, the second correction information,and the weighting factor α. The controller 240 causes the display 200 todisplay a reference image corrected using the generated third correctioninformation. The reference image is an image prepared in advance fordetermining the weighting factor α. Any image may be adopted as thereference image. For example, the reference image may include a square,circle, a plurality of lines arranged in a grid, or the like.

The controller 240 changes the weighting factor α in accordance withoperation by the subject 130 who observed the reference image displayedas the first virtual image 140 a, for example. When the weighting factorα is changed, the controller 240 can newly generate the third correctioninformation based on the first correction information, the secondcorrection information, and the changed weighting factor α. Thecontroller 240 causes the display 200 to display a reference imagecorrected using the newly generated third correction information. Inaccordance with operations, the weighting factor α may be changed, andthe reference image that is corrected using the newly generated thirdcorrection information may be displayed, a plurality of times. Thecontroller 240 can finalize the weighting factor α and store theweighting factor α in the memory 230 in accordance with an operation bythe subject 130, for example.

When, for example, the subject 130 drives the vehicle 100 after theweighting factor α is finalized as above, the controller 240 can use theposition of both eyes of the subject 130 to dynamically correct thedisplay image to be displayed by the display 200. Specifically, thecontroller 240 can generate the third correction information based onthe first correction information and the second correction informationcorresponding to the position of both eyes of the subject 130 and thefinalized weighting factor α. The controller 240 can dynamically correctthe display image displayed on the display 200 using the firstcorrection information and the second correction information. Thecontroller 240 corrects the parallax barrier 210 based on the determinedthird correction information. The display image may include anyinformation or image. For example, the display image may include adriving support image with the travel speed of the vehicle 100, thepredicted route, the speed limit of the road being traveled, signinformation, a route guide, or the like; an image indicating obstaclessuch as pedestrians; or another such image.

The weighting factor α that allows the subject 130 to perceive the leastamount of distortion in the virtual image 140 differs depending on thevideo composition ratio of the dominant eye and the non-dominant eye ofthe subject 130. By changing the weighting factor α while viewing thevirtual image 140 of the reference image with both eyes, for example,the subject 130 can cause the weighting factor α that produces the leastdistortion of the virtual image 140 to be stored in the memory 230. Thisconfiguration allows the display image to be corrected using a weightingfactor α that takes into consideration the dominant eye of the subject130. The distortion of the virtual image 140 perceived by the subject130 is therefore reduced, improving visibility of the virtual image 140.This improves the convenience of a technique for making the virtualimage 140 visible to the subject 130.

The controller 240 can correct the right eye image based on the firstcorrection information. The controller 240 causes the subpixels formingthe right subpixel groups Pgr to display the corrected right eye image.The controller 240 can correct the left eye image based on the secondcorrection information. The controller 240 can cause the subpixelsforming the left subpixel groups Pgl to display the corrected left eyeimage.

The controller 240 can correct the parallax barrier 210 based on thethird correction information. Specifically, based on the thirdcorrection information, the controller 240 can control the liquidcrystal shutter forming the parallax barrier 210. In greater detail, thecontroller 240 can control the liquid crystal shutter so as to changethe portions forming the light-blocking surfaces 210 a of the parallaxbarrier 210.

Operations of the three-dimensional projection apparatus 120 todetermine the above-described weighting factor α are described withreference to the flowchart in FIG. 12. These operations may be performedbefore the subject 130 drives the vehicle 100, for example.

Step S400: the controller 240 acquires information related to theposition of the eyes of the subject 130 from the detection apparatus 110through the communication interface 220.

Step S401: the controller 240 determines the first correctioninformation corresponding to the position of the right eye and thesecond correction information corresponding to the position of the lefteye of the subject 130.

Step S402: the controller 240 sets the weighting factor α to an initialvalue.

Step S403: the controller 240 generates the third correction informationbased on the first correction information, the second correctioninformation, and the weighting factor α.

Step S404: the controller 240 corrects the right reference image basedon the first correction information. The controller 240 corrects theleft reference image based on the second correction information.

Step S405: the controller 240 causes the subpixels forming the rightsubpixel groups Pgr to display the corrected right reference image. Thecontroller 240 causes the subpixels forming the left subpixel groups Pglto display the corrected left reference image.

Step S406: the controller 240 corrects the parallax barrier 210 based onthe third correction information.

Step S407: upon detecting operation by the subject 130, for example, thecontroller 240 determines whether to change the weighting factor α. Whenthe controller 240 determines to change the weighting factor α (stepS407: Yes), the process proceeds to step S408. Conversely, when thecontroller 240 determines not to change the weighting factor α (stepS407: No), the process proceeds to step S409.

Step S408: the controller 240 changes the weighting factor α inaccordance with the operation in step S407, for example. Subsequently,the process returns to step S403.

Step S409: the controller 240 finalizes the weighting factor α inaccordance with the operation in step S407, for example, and stores theweighting factor α in the memory 230. The process then terminates.

Operations of the three-dimensional projection apparatus 120 fordynamically correcting the display image in accordance with the positionof both eyes of the subject 130 are now described with reference to FIG.13 for a configuration adopting the above-described first algorithm.These operations may be repeatedly performed while the subject 130 isdriving the vehicle 100, for example.

Step S500: the controller 240 acquires information related to theposition of the eyes of the subject 130 from the detection apparatus 110through the communication interface 220.

Step S501: the controller 240 determines the first correctioninformation corresponding to the position of the right eye and thesecond correction information corresponding to the position of the lefteye of the subject 130.

Step S502: the controller 240 generates the third correction informationbased on the first correction information, the second correctioninformation, and the weighting factor α that is stored in the memory230.

Step S503: the controller 240 corrects the right eye image based on thefirst correction information and corrects the left eye image based onthe second correction information.

Step S504: the controller 240 causes the subpixels forming the leftsubpixel groups Pgl to display the corrected left eye image and causesthe subpixels forming the right subpixel groups Pgr to display thecorrected right eye image.

Step S505: the controller 240 corrects the parallax barrier 210 based onthe third correction information. The process then terminates.

In accordance with the position of the eyes of the subject 130, thethree-dimensional projection apparatus 120 of the present embodimentsthus corrects the parallax barrier 210 and the image that the displaysurface 2010 is caused to display. The distortion, corresponding to theposition of the eyes of the subject 130, that may occur in the firstvirtual image 140 a and the second virtual image 140 b due to the shapeof the vehicle 100 is therefore corrected.

Accordingly, the subject 130 can see the virtual image 140 of athree-dimensional image with reduced distortion.

The three-dimensional projection apparatus 120 of the presentembodiments corrects the right eye image based on the first correctioninformation corresponding to the position of the right eye and correctsthe left eye image based on the second correction informationcorresponding to the position of the left eye. The subject 130 cantherefore see images, corresponding to the position of the eyes, thathave reduced distortion. Accordingly, the subject 130 can see thevirtual image 140 of a three-dimensional image with reduced distortion.

The three-dimensional projection apparatus 120 according to the presentembodiments corrects the parallax barrier 210 based on the thirdcorrection information generated using the first correction information,the second correction information, and the weighting factor α. Theparallax barrier 210 seen by both eyes of the user is corrected based onthe positions of the right eye and the left eye. The right eye image andleft eye image with reduced distortion are therefore each blocked ortransmitted by the parallax barrier 210, which has reduced distortion.Accordingly, the right eye of the subject 130 can see a right eye imagewith reduced distortion. The left eye can see a left eye image withreduced distortion.

The three-dimensional projection apparatus 120 according to the presentembodiments determines the weighting factor α based on the dominant eyeof the subject 130. Images are therefore corrected using a weightingfactor α that takes into consideration the dominant eye of the subject130. Accordingly, the distortion of the virtual image 140 perceived bythe subject 130 is reduced, improving visibility of the virtual image140.

The present disclosure is based on drawings and embodiments, but itshould be noted that a person of ordinary skill in the art could easilymake a variety of modifications and adjustments based on the presentdisclosure. Therefore, such changes and modifications are to beunderstood as included within the scope of the present disclosure. Forexample, the functions and the like included in the various means,steps, and the like may be reordered in any logically consistent way.Means, steps, or the like may be combined into one or divided.

For example, in the configuration described in the above embodiments,correction information including information indicating the shift amountof each of a plurality of feature points on an image displayed by thedisplay 23 or 200 is used to deform the image, thereby correcting theimage. The content of the correction information and the processing tocorrect images are not, however, limited to the above-describedconfiguration.

For example, the controller 26 or 240 may project the image for thedisplay 23 or 200 to display onto a polygon as a texture image. Anyalgorithm to project a texture image onto a polygon may be adopted. Forexample, a texture image may be projected onto a polygon using mappinginformation indicating the correspondence relationship between texturecoordinates on the texture image and the vertices of the polygon. Theshape of the polygon may be determined freely. The mapping informationmay be determined freely. The controller 26 or 240 arranges the polygonwith the texture image projected thereon within three-dimensionalvirtual space, for example. The controller 26 or 240 causes the display23 or 200 to display an image of a region, located within the virtualspace that includes the polygon, as viewed from the perspective of avirtual camera. Like the above-described embodiments, this configurationallows the subject 13 or 130 to see the virtual image 14 or 140 of theimage that the display 23 or 200 is caused to display.

In the above-described configuration using a polygon, the correctioninformation may include information for changing the mapping informationto reduce distortion of the shape of the virtual image 14 or 140. Forexample, the correction information may include information for changingthe vertices of the polygon that correspond to texture coordinates onthe texture image to other vertices. For example, the texture imageprojected onto the polygon can be deformed evenly or unevenly bycorrection based on the correction information and then displayed by thedisplay 23 or 200. Like the above-described embodiments, the imageprojection light of the texture image deformed based on the correctioninformation allows the subject 13 or 130 to see a virtual image 14 or140 with reduced distortion. The processing to change the mappinginformation can be executed at high speed when, for example, thecontroller 26 or 240 includes dedicated hardware. The dedicated hardwaremay, for example, include a graphics accelerator. This configurationallows high-speed correction of the image that the display 23 or 200 iscaused to display. Therefore, when the position of both eyes of thesubject 13 or 130 changes, for example, the distortion of the virtualimage 14 or 140 due to the change in the position of both eyes can bereduced at high speed. This further improves the convenience of atechnique for making the virtual image 14 or 140 visible to the subject13 or 130.

In the above-described configuration using a polygon, the correctioninformation may include information for changing the shape of thepolygon to reduce distortion of the shape of the virtual image 14 or140. For example, the texture image projected onto the polygon can bedeformed evenly or unevenly by correction based on the correctioninformation and then displayed by the display 23 or 200. Like theabove-described embodiments, the image projection light of the textureimage deformed based on the correction information allows the subject 13or 130 to see a virtual image 14 or 140 with reduced distortion. Theprocessing to change the shape of the polygon can be executed at highspeed when, for example, the controller 26 or 240 includes dedicatedhardware. The dedicated hardware may, for example, include a graphicsaccelerator. This configuration allows high-speed correction of theimage that the display 23 or 200 is caused to display. Therefore, whenthe position of both eyes of the subject 13 or 130 changes, for example,the distortion of the virtual image 14 or 140 due to the change in theposition of both eyes can be reduced at high speed. This furtherimproves the convenience of a technique for making the virtual image 14or 140 visible to the subject 13 or 130.

In the configuration described in the first embodiments, one image isdisplayed by the display 23. The number of images displayed by thedisplay 23, however, may be determined freely. The positions, shapes,and sizes of the images displayed by the display 23 may be determinedfreely. For each image that the controller 26 in this configurationcauses the display 23 to display, the controller 26 may dynamicallycorrect the image in accordance with the position of both eyes of thesubject 13. Each image that the display 23 is caused to display may becorrected in the same way as in the above-described embodiments. Forexample, for each image that the display 23 is caused to display, thememory 25 stores a plurality of pieces of correction informationcorresponding to a plurality of reference positions in actual space. Thecontroller 26 may determine the weighting factor α and the thirdcorrection information using the above-described first algorithm orsecond algorithm, for example, for each image that the display 23 iscaused to display. For each image that the display 23 is caused todisplay, the controller 26 corrects the image using the third correctioninformation corresponding to the image. This configuration allows thesubject 13 to see a plurality of virtual images 14 respectivelycorresponding to the plurality of images. This further improves theconvenience of a technique for making the virtual image 14 visible tothe subject 13.

The invention claimed is:
 1. A three-dimensional projection apparatuscomprising: a display including a display surface having a plurality ofsubpixels arranged in a grid along a first direction and a seconddirection substantially orthogonal to the first direction; an opticalelement configured to define a light beam direction of an image lightemitted from the subpixels for each strip-shaped region of a pluralityof strip-shaped regions extending in the second direction on the displaysurface; an optical member configured to project the image light, thelight beam direction of which is defined by the optical element, so thata virtual image of the display surface is formed; and a controllerconfigured to: acquire information related to a position of a first eyeand a second eye of a subject based on a position of a face of thesubject, an orientation of the face, or a size of the face, generatefirst correction information, the first correction informationindicating a shift amount of a feature point in a first imagecorresponding to a position of the first eye, generate second correctioninformation, the second correction information indicating a shift amountof a feature point in a second image corresponding to a position of thesecond eye, generate third correction information, the third correctioninformation indicating a weighted shift amount, by applying a weightingfactor to the first correction information and the second correctioninformation, move an image to be displayed by the display surface by theweighted shift amount, and control a liquid crystal shutter of theoptical element to change portions forming light-blocking surfaces ofthe optical element in accordance with the third correction information.2. The three-dimensional projection apparatus of claim 1, wherein thecontroller is configured to determine the weighting factor based on adominant eye of the subject.
 3. The three-dimensional projectionapparatus of claim 1, further comprising a memory configured to store aplurality of pieces of correction information corresponding to aplurality of reference positions in actual space, wherein the controlleris configured to: select the first correction information, or generatethe first correction information based on two or more pieces ofinformation of a plurality of pieces of correction information; andselect the second correction information, or generate the secondcorrection information based on two or more pieces of information of theplurality of pieces of correction information.
 4. The three-dimensionalprojection apparatus of claim 1, wherein the display surface isconfigured to display an image of a polygon as viewed from a virtualcamera, the polygon having a texture image projected thereon usingmapping information, and wherein the controller is configured to correctthe polygon by changing the mapping information in accordance with aposition of the both eyes of the subject.
 5. The three-dimensionalprojection apparatus of claim 4, wherein the controller is configured tocorrect the polygon by changing mapping information indicating acorrespondence relationship between texture coordinates on the textureimage and a plurality of vertices of the polygon.
 6. Thethree-dimensional projection apparatus of claim 4, wherein thecontroller is configured to correct the polygon by changing a shape ofthe polygon.
 7. A three-dimensional projection system comprising: adetection apparatus configured to detect a position of a first eye and asecond eye of a subject based on a position of a face of the subject, anorientation of the face, or a size of the face; and a three-dimensionalprojection apparatus comprising: a display including a display surfacehaving a plurality of subpixels arranged in a grid along a firstdirection and a second direction substantially orthogonal to the firstdirection, an optical element configured to define a light beamdirection of an image light emitted from the subpixels for eachstrip-shaped region of a plurality of strip-shaped regions extending inthe second direction on the display surface, an optical memberconfigured to project the image light, the light beam direction of whichis defined by the optical element, so that a virtual image of thedisplay surface is formed, and a controller configured to: acquireinformation related to the position of the first eye and the second eyefrom the detection apparatus, generate first correction information, thefirst correction information indicating a shift amount of a featurepoint in a first image corresponding to a position of the first eye,generate second correction information, the second correctioninformation indicating a shift amount of a feature point in a secondimage corresponding to a position of the second eye, generate thirdcorrection information, the third correction information indicating aweighted shift amount, by applying a weighting factor to the firstcorrection information and the second correction information, move animage to be displayed by the display surface by the weighted shiftamount, and control a liquid crystal shutter of the optical element tochange portions forming light-blocking surfaces of the optical elementin accordance with the third correction information.
 8. A vehiclecomprising a three-dimensional projection system comprising: a detectionapparatus configured to detect a position of a first eye and a secondeye of a subject based on a position of a face of the subject, anorientation of the face, or a size of the face; and a three-dimensionalprojection apparatus comprising: a display including a display surfacehaving a plurality of subpixels arranged in a grid along a firstdirection and a second direction substantially orthogonal to the firstdirection, an optical element configured to define a light beamdirection of an image light emitted from the subpixels for eachstrip-shaped region of a plurality of strip-shaped regions extending inthe second direction on the display surface, an optical memberconfigured to project the image light, the light beam direction of whichis defined by the optical element, so that a virtual image of thedisplay surface is formed, and a controller configured to: acquireinformation related to the position of the first eye and the second eyefrom the detection apparatus, generate first correction information, thefirst correction information indicating a shift amount of a featurepoint in a first image corresponding to a position of the first eye,generate second correction information, the second correctioninformation indicating a shift amount of a feature point in a secondimage corresponding to a position of the second eye, generate thirdcorrection information, the third correction information indicating aweighted shift amount, by applying a weighting factor to the firstcorrection information and the second correction information, move animage to be displayed by the display surface by the weighted shiftamount, and control a liquid crystal shutter of the optical element tochange portions forming light-blocking surfaces of the optical elementin accordance with the third correction information.
 9. Thethree-dimensional projection apparatus of claim 1, wherein, thethree-dimensional projection apparatus is further configured to storecorrespondence information that associates the position of the face ofthe subject, the orientation of the face, or the size of the face withthe position of the first eye and the second eye of the subject, and thecontroller acquires the information related to the position of the firsteye and the second eye of the subject based on the correspondenceinformation.
 10. The three-dimensional projection apparatus of claim 9,wherein the correspondence information is a lookup table that is storedin advance.
 11. The three-dimensional projection apparatus of claim 9,wherein the controller acquires the information related to the positionof the first eye and the second eye of the subject by, detecting theposition of the face of the subject, the orientation of the face, or thesize of the face, detecting a position of the first eye and the secondeye of the subject in actual space, and associating the position of theface of the subject, the orientation of the face, or the size of theface with the position of the first eye and the second eye of thesubject in actual space using the correspondence information to acquirethe information related to the position of the first eye and the secondeye of the subject.
 12. The three-dimensional projection system of claim7, wherein, the three-dimensional projection system further comprisescorrespondence information that associates the position of the face ofthe subject, the orientation of the face, or the size of the face withthe position of the first eye and the second eye of the subject, and thedetection apparatus is further configured to detect the position of thefirst eye and the second eye of the subject based on the correspondenceinformation.
 13. The three-dimensional projection system of claim 12,wherein the correspondence information is a lookup table that is storedin advance.
 14. The three-dimensional projection system of claim 12,wherein the detection apparatus is further configured to detect theposition of the first eye and the second eye of the subject by,detecting the position of the face of the subject, the orientation ofthe face, or the size of the face, detecting a position of the first eyeand the second eye of the subject in actual space, and associating theposition of the face of the subject, the orientation of the face, or thesize of the face with the position of the first eye and the second eyeof the subject in actual space using the correspondence information toacquire the information related to the position of the first eye and thesecond eye of the subject.
 15. The vehicle of claim 8, wherein, thethree-dimensional projection system further comprises correspondenceinformation that associates the position of the face of the subject, theorientation of the face, or the size of the face with the position ofthe first eye and the second eye of the subject, and the detectionapparatus is further configured to detect the position of the first eyeand the second eye of the subject based on the correspondenceinformation.
 16. The vehicle of claim 15, wherein the correspondenceinformation is a lookup table that is stored in advance.
 17. The vehicleof claim 15, wherein the detection apparatus is further configured todetect the position of the first eye and the second eye of the subjectby, detecting the position of the face of the subject, the orientationof the face, or the size of the face, detecting a position of the firsteye and the second eye of the subject in actual space, and associatingthe position of the face of the subject, the orientation of the face, orthe size of the face with the position of the first eye and the secondeye of the subject in actual space using the correspondence informationto acquire the information related to the position of the first eye andthe second eye of the subject.